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ABSTRACT- This Project is an android application 
enabled with IOT and ASL. Which aids the people with 
visual and hearing impairments to learn a new language? 
Initially the application will prompt the user to select how 
he is differently abled (deaf or blind).Now based on the 
option selected for blind people the entire UI will be 
converted into voice recognition system and text to Speech 
,Hence the user can orally listen and learn and to learn to 
write, a special interface called Braille Keyboard is used 
..Three foreign languages are included in this project 
namely English French and German. The Alphabets along 
with the Braille codes will be taught to the blind people. 
The Main Novelty of this project is that a Braille keyboard 
is interfaced with the application. If the user is deaf, The 
UI will become touch based .Now the user will start 
learning visually and he will understand the words through 
ASL .After each lesson the user will be tested on their 
understanding. The entire features of the application is 
designed based on the requirements gathered from the 
blind and deaf students by interacting with them in person . 
So the main advantage of this application is that it will be 
user friendly for the actual real time end user. Every lesson 
can be replayed any number of times until the user 
understands it thoroughly phones. The application assists 
the blind people and make their learning process easier 
with voice interface and in addition a Braille keyboard is 
also being used .They will be given a clearer vision on the 
Braille code for each and every alphabet The application 
assists the hearing impaired people to learn a new language 
by the standard sign language , the ASL. 
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I. INTRODUCTION 


This project is based on social service app which is 
especially for visually and hearing-impaired people. There 
are about 285 million people in the world are visually 
impaired, of whom 39 million are blind and 246 million 
have moderate visual impairment. There are about 37 
million people across the globe who are blind, over 15 
million are from India. Around 466 million people 
worldwide have disabling hearing loss, and 34 million of 
these are children. It is estimated that by 2050 over 900 
million people will have disabling hearing loss. All the 
language learning apps available are being used only by the 
normal people .People with disabilities find it difficult to 
use these apps .One main disadvantage of the existing 
applications is that they don’t have voice control .[10]The 
main vision of this project is to make people with 
disabilities to use it with ease . Since we develop this 
project as a Mobile application, this project provides an 
interactive system for the visually and hearing impaired. 


Il. RELATED WORK 


In the existing system, there are no applications 
developed to aid visually impaired people to learn a new 
language based on voice interface [1]. They have to use the 
applications which have touch interface and do not have 
voice input facility. For deaf there are some applications 
based on ASL .But the pictures used for the signs are not 
very clear and some features are not free the user has to 
pay to use those features.[4]. Moreover there is no such 
application which enables both the visually and hearing 
impaired people to use a single application. 


DISADVANTAGES 

e In the applications available for the deaf, the signs 
used are not very clear and also they are not fully 
free[3]. 

e The normal language learning applications does not 
include the Braille codes for the alphabets which are 
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the only way a visually impaired person learn a 
language. 

e Also so Far no application allows the Braille keyboard 
to be interfaced inside a mobile application [9]. 


e The Braille keyboards are meant only for Desktop 
PC’s. 


Ill. PROPOSED WORK 


In the proposed system, we develop a mobile application 
to rectify the problems in the existing system. Using this 
mobile application, the blind people can easily learn a new 
language along with their Braille codes through easy voice 
interface .Also the hearing impaired people can learn the 
language along with its Signs based on American Sign 
Language a standard form accepted widely. 

The work flow of our project is as follows: Initially the 
app will prompt the user to select how he is differently 
abled (deaf or blind)[5]. Now based on the option selected 
for blind people the entire UI will be converted into voice 
recognition system and text to speech, Hence the user can 
orally listen and learn and to learn to write, a special 
interface called Braille Keyboard is used. Three foreign 
languages are included in this project namely English 
French and German. The Alphabets along with the Braille 
codes will be taught to the blind people. The Main Novelty 
of this project is that a Braille keyboard is interfaced with 
the app. 

If the user is deaf, the UI will become touch based .Now 
the user will start learning visually and he will understand 
the words through American Sign Language .After each 
lesson the user will be tested on their understanding. The 
entire feature of the app is designed based on the 
requirements gathered from the blind and deaf students by 
interacting with them in person. 

In this application the person will be tested at the end of 
every lesson .The app will ask the user to enter the Braille 
code for one of the alphabet they learnt in that lesson. The 
user can give the input through the Braille Keyboard which 
will be verified and prompt the user whether it is correct or 
incorrect[6]. 

The Hearing impaired person will also be tested at the 
end of every lesson based on the signs they learnt in the 
corresponding lesson. 
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Fig.1: Architecture diagram of the proposed system 


ADVANTAGES 


e In proposed system, it is made easy for the visually 
impaired people to learn a new language through a 
mobile application 

e They are also allowed to use a Braille Keyboard which 
they are very adept in using to interact with Desktop 
PC’s 

e There are no mobile application so far which allows the 
use of Braille keyboard 

e Every lesson can be replayed any number of times and 
page navigation is also only through voice input 

e The application includes three foreign Languages 
Namely English, French and German. 

e Both the visual impaired and hearing impaired person 
can use a single application. 


IV. MODULES 


A) Unique Registration and Login 

The Blind and deaf people will be asked to enter their 
name through voice input for blind people and through 
normal keyboard typing for deaf .Next the type of disability 
(Blind or Deaf ) should be mentioned .AIl the details 
entered will be stored in a database[2]. 
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Fig 2: Registration Module 


B) Language Selection 


If the Type is blind, the next page will ask the user to 
select the language they want to learn .There are three 
options available namely English French and German 

Every Language will be split into four—five lessons and 
the user can navigate through each lesson using “next” and 
“previous” keywords. 


= Languages 


Fig 3: Language Selection Module 


C) Learning Lessons 


Every language will have four-five lessons and each 
lesson will contain 6 alphabets along with their Braille 
codes. Every alphabet can be played any number of times 
according to the grasping power of the user 

At the end of every lesson a quick revision will be 
available to have a quick glance of everything seen in that 
lesson so far[8]. 
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Fig 4: Lesson Learning Module (Blind) 


D) Blind (Test) 

This is the main novelty of our project .In this module the 
user will be tested on whatever they learnt in the lesson so 
far .The app will ask the user to enter the Braille code for 
any alphabet (from the list they learnt ) [7]. 

The input will be given through the Braille Keyboard 
.The user will enter the code through the Braille keyboard 
which will be verified by the app and say whether it is 
correct or not. 


= English_Testing! 


Fig 5: Blind Test Module 


E) American Sign Language (ASL) 

If the user type is deaf, the next page will have a small 
introduction about ASL which is widely used by hearing 
impaired people to learn a new language [12]. 

From here the user can navigate through normal buttons. 
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Fig 6: ASL Module 


F) Learning Lessons 

The next page will contain the alphabets split into four 
lessons. Each lesson will have 8 alphabets along with their 
signs .At the end of every lesson a test will be conducted to 
the user [11]. 


= D_lessont1 


Fig 7: Lesson Learning Module (Deaf) 


G) Deaf (Test) 


In the test for deaf .A sign will be given and the user will 
be asked to select the correct alphabet for that sign and 
another way of test is where the app asks the user to 
correctly select the sign for the given letter. 


Match the correct answer 


Pl 


S ka 


Fig 8: Deaf test module 


Vv. CONCLUSION 


This application helps the blind people in learning the 
alphabets of languages like English, French and German 
along with their respective Braille codes. This app is 
designed in such a way that the user feels it easy to use and 
operate as the app is mostly controlled by voice 
.Additionally a Braille keyboard is designed for the user to 
test the codes learnt. And for the Deaf, ASL is taught along 
with test for individual lessons. 

This is project was made based on the survey made on 
the existing apps for the differently-abled people and the 
suggestions given by the (differently abled mobile phone 
users). The usability, actability and user-centered aspects 
were considered during the development. Thus it is 
designed in such a way overcoming all the disadvantages 
stated. 


VI. FUTURE ENHANCEMENT 


Future enhancements can be differentiating the type of 
user either beginner or advanced and categorizing the 
lessons according to the type. 

For Blind - The project can be made into a fully voice 
controlled system with a wireless Braille keyboard which 
can input more number of characters. 

For Deaf — ASL for words and sentences can be introduced 
along with GIF’s or video format, a complete language 
learning guide for the differently- abled people can be done. 


REFERENCES 


[1] Adam Dabrowski, Damian MHuderek, Marcin 
Iwanowski, Piotr Kardys, “A new android application 
for blind and visually impaired people’, IEEE 
Transaction on Computers, Poland 2016. 


Copyright © 2020. Innovative Research Publication. All Rights Reserve 387 


International Journal of Innovative Research in Computer Science & Technology (IJIRCST) 


[2] R. Guha, V. Gupta, V. Raghunathan, R. Srikant, “User 
modelling for a personal assistant”, The 8th WSDM 
International Conference, Shanghai 2015. 

[3] S. M. Kulkarni, Rupa N. Digole, ’Smart navigation 
system for visually impaired person’, International 
Journal of Advanced Research in Computer and 
Communication Engineering, India 2015. 

[4] Pooja Sharma, Shimi S. L, “Design and Development 
of Virtual Eye for the blind”, International Journal of 
Innovative Research in Electrical, Electronics, 
Instrumentation and Control Engineering, India 2015. 

[5] J. Hildenbrand, “What is Google TalkBack”, 
AndroidCentral.com, 2014. 

[6] Bairoju Vishwa Rupa Chary, B.Santosh Kumar, 
“Rescue system for visually impaired blind persons”, 
International Journal of Engineering Trends and 
Technology (JETT), India 2014. 

[7] Harshad Girish Lele, Mrunmayi Mohan Modak, Viten 
Vilas Lonkar, Varun Vasant Marathe, “Electronis path 
guidance for visually impaired people’, The 
International Journal Of Engineering And Science 
(JES), India 2013. 

[8] Abhishek Srivastava, Adhar Vashishth, Akshay 
Sharma, “An assistive reading system for visually 
impaired using OCR and TTS”, International Journal 
of Computer Applications, India 2014. 

[9] Amanda Hastings, Ravi Kuber, Matthew Tretter, 
“Determining the accessibility of mobile screen readers 
for blind users”, IASTED Conference on Human- 
Computer Interaction, Baltimore, USA 2012. 

[10]Shreyashi Narayan Sawant " Sign Language 
Recognition System to aid Deaf-dumb People Using 
PCA" International Journal of Computer Science & 
Engineering Technology (IJCSET) 

[11]Dalia Nashat, Abber Shoker, Reem AI-Ebilan “An 
Android application to aid uneducated deaf-dumb 
people” International Journal of Computer Science and 
Mobile Applications (IJCSMA) 

[12] Majzoob Kamal, Aldein Omer, Mohmed Sirelkhtem 
Adrees, Osama E. Sheta “Alternative Central Mobile 
Application Strategy to Deaf and Dumb Education in 
Third World Countries” International Journal of Soft 
Computing and Engineering (IJSCE) 


Copyright © 2020. Innovative Research Publication. All Rights Reserve 


ISSN: 2347-5552 Volume- 8, Issue- 6, November 2020 
https://doi.org/10.21276/ijircst.2020.8.6.3 
www. ijircst.org 


388 


